A Benchmark for Model Matching Systems: The Heterogeneous Metamodel Case

Recently, several model matching tools emerged that compute the corresponding elements of two different models. Having a closer look at concrete matching scenarios, one may find different cases such as matching different versions of one model, e.g., model evolution, or matching heterogenous models that do not have a common predecessor. Especially, the latter case is interesting as a first step when similar but at the same time potentially heterogenous metamodels have to be bridged for exchanging models between different tools. Most model matching tools provide means for matching heterogenous metamodels. In particular, several heuristics are implemented in matching algorithms to reason about heterogenous structures and terminologies used in the metamodels to be matched. However, which set of heuristics is appropriate for heterogeneous metamodel matching and how to weight them when aggregating the different results to achieve correspondences with high quality is still an open challenge. To address this issue, we propose a benchmark consisting of real world metamodels and manually defined expected correspondences that allows to evaluate automatically the quality of the output of model matching tools.